Methods and systems for segmentation and surface matching

ABSTRACT

A contoured surface map of a lesion within a patient is obtained by shifting a reference surface to an estimated location in operational images. The process can be repeated to minimize errors, and the contoured surface map can then be segmented.

TECHNICAL FIELD

This invention relates to methods and systems for verifying anatomicalfeatures of a patient undergoing radiation therapy and, moreparticularly, to methods and systems for using reference images toidentify surface elements of anatomical elements imaged at a later time.

BACKGROUND INFORMATION

Radiation-emitting devices are used for the treatment of canceroustumors within patients. The primary goal of treating cancerous tumorswith radiation therapy is the complete eradication of the cancerouscells, while the secondary goal is to avoid, to the maximum possibleextent, damaging healthy tissue and organs in the vicinity of the tumor.Typically, a radiation therapy device includes a gantry that can berotated around a horizontal axis of rotation during the delivery of atherapeutic treatment. A particle linear accelerator (“LINAC”) islocated within the gantry, and generates a high-energy radiation beam oftherapy, such as an electron beam or photon (x-ray) beam. The patient isplaced on a treatment table located at the isocenter of the gantry, andthe radiation beam is directed toward the tumor or lesion to be treated.

Radiation therapy typically involves a planning stage and a treatmentstage. Generally, the planning stage involves acquiring images of alesion (using, for example an x-ray device) and subsequently using theimage(s) to accurately measure the location, size, contour, and numberof lesions to be treated. These are used to establish certain treatmentplan parameters, such as an isocenter, beam angles, energy, aperture,dose distribution, and other parameters in an attempt to irradiate thelesion while minimizing damage to surrounding healthy tissue.

Imaging is often used by oncologists in determining the treatmentparameters of radiation therapy plans such that the prescribed radiationis sufficient to eliminate the cancerous cells and while conforming theshape of the dose distribution to a target volume to the greatest extentpossible, thereby sparing healthy tissue from exposure to potentiallyharmful doses of radiation. To develop a preferred treatment plan,simulations can be performed to design a set of beams which accomplishthis goal that calculate the dose at each point in the patient resultingfrom this set of beams. The dose distribution can be represented, forexample, as isodose lines or as three-dimensional isodose surfaceswithin the patient. The treatment goal is to encompass the lesion and anappropriate safety margin within the 100% isodose surface. The treatmentplan is then administered, usually at a later date and over a period ofweeks, based on the treatment parameters. One shortcoming of thisapproach is that the time lapse between treatment planning and treatmentdelivery allows for changes to the patient's anatomy, therebypotentially rendering the treatment plan sub-optimal. Changes such aslesion movement, growth, organ shifting, or other morphisms can causehealthy tissue to become subject to potentially harmful radiation, andcancerous tissue to extend beyond the boundaries of the originaltreatment plan.

Once a treatment plan is determined, the patient receives the radiationtreatments during a number of sessions (fractions). Treatment oftenincludes significant time lapses between individual fractions and canalso span many weeks (e.g., once a day five days a week for four weeks.)Because organs can change location and/or shape from the time ofplanning to the delivery of the initial fraction, as well as fromfraction to fraction, the original segmented, contoured image may nolonger accurately represent the lesion being treated. As a result, thetreatment plan may no longer be optimal. Three-dimensional imagingmodalities that are able to discern soft-tissues are therefore used inthe treatment room in order to detect and compensate for organ motion.However, because of the time constraints imposed during the individualfractions, and the lack of a trained physician during the fractions, itmay not be possible to generate an updated segmented or contoured imageof the lesion. Thus, methods that provide fast, accurate, and reliableimages and patient positioning data without requiring a physician'sexpertise are of great benefit to a radiation technologist administeringthe radiation treatment.

Therefore, a fast technique to segment organs or structures of interestprior to a medical procedure with minimal user guidance is needed.

SUMMARY OF THE INVENTION

The present invention provides systems and methods to obtain asegmented, contoured, three-dimensional representation of an anatomicalfeature or structure of interest of a patient prior to a medicalprocedure with minimal user guidance, which uses the informationprovided by pre-operation contours approved by the physician as aninitial estimate of the treatment contour shape. Such techniquesfacilitate determining patient positioning corrections to compensate fordisplacement and morphological change, modifying treatment parameters toaccount for the current location of the lesion, and/or calculatingchanges in lesion size and shape over the course of a medical treatment.The invention utilizes a surface model derived during the treatmentplanning phase and images taken during the treatment phase. In general,the invention relates to using a previously contoured image of a tissuevolume acquired during the treatment planning phase as an initialestimate of the current surface.

In a typical embodiment, a user shifts, rotates and/or scales theplanning contour (using, for example, a video screen) until the contourfits as close as possible to the lesion or organ in the treatment image,and then uses this shifted contour as an initial estimate for a localsegmentation algorithm. The local segmentation algorithm, in turn, usesthis estimate as a starting point to find edges in the current treatmentimage, resulting in a segmented, contoured surface of the lesion in thecurrent image. The invention is based on an assumption that the volumein the current image to be segmented has translated, rotated and/orscaled from its position when initially imaged, and that the planningcontour can serve as a reference surface for the local segmentationprocess. The surface is moved to the correct location in the currentimage (either manually and/or automatically) by using iterations oflocal segmentation and surface-matching techniques to gradually move theplanning contour to the correct location in the treatment image, whichcan then used as an initial estimate for a final iteration of a localsegmentation process.

In the radiotherapy application, for example, adjustments to thetreatment parameters and/or patient positioning can be made based on thenewly segmented volume image, and thus the invention facilitates rapidand accurate treatment position adjustment just prior to treatmentdelivery while including important clinical and/or practical concernsnot addressed by other conventional methods.

In one aspect, a method for obtaining a contoured image of a lesionwithin a patient (for the purposes of administering radiation treatment,for example) includes providing a contoured reference image of thelesion based on images of the lesion taken at a first time (such asduring a treatment planning session), generating a set of operationalimages at a second time (such as during a treatment delivery session),shifting (e.g., translating, rotating, scaling, or any combinationthereof) the reference surface to an approximated location in theoperational images, segmenting the operational images based on theshifted reference surface (using, for example, a local segmentationtechnique), and determining a contoured surface of the lesion at thesecond time based on the segmentation.

The reference surface and/or operational images can be created fromimages generated using any suitable tomographic or other imagingmodality, e.g., a CT scanner, a three-dimensional ultrasound device, aPET scanner, or an MRI device. The approximated location can bedetermined manually, or by indicating one or more points on theoperational images and using a surface mapping algorithm to map surfaceelements on the reference surface to the indicated points. The surfaceelements on the reference surface can include triangles or othertwo-dimensional shapes, lines or points, and the points on theoperational images can represent edge boundaries. In some embodiments,two-dimensional cross-sectional cuts through the surfaces and/or images(either radially offset or parallel to each other) and can be used asinput into a segmentation algorithm. The process of shifting thereference surface with surface matching and segmentation can be repeatediteratively using the shifted reference surface in place of theoperational images. A minimization threshold (e.g., minimum squareddistance) can be used to determine when the shifted image is optimallyplaced.

Weights can be assigned to the surface elements on the reference and/ortreatment surface, and can be based on a degree of certainty that thesurface element corresponds to a particular feature of the lesion, whichin some cases can be an edge of the lesion; and/or on the clinicalimportance of an anatomical feature represented by the surface elementand, in some embodiments, the proximity of an anatomical featurerepresented by the surface element to another anatomical structure ofthe patient. In some embodiments, the weights can be based on thedensity of the surface elements within a particular area of the image,and/or the area of the surface element itself.

In another aspect, a system for obtaining a contoured image of a lesionincludes a register for storing a reference surface of the lesion basedon images acquired at a first time, a second set of images of the lesiontaken at a second time, and estimated surface elements on the second setof images. The system also includes a mapping module for mapping surfaceelements on the reference surface to estimated surface elements on theset of images and a processor for shifting (e.g., translating, rotating,and/or scaling) the reference surface based on the matched surface.

The foregoing and other objects, features and advantages of the presentinvention disclosed herein, as well as the invention itself, will bemore fully understood from the following description of preferredembodiments and claims, when read together with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to the sameparts throughout the different views. Also, the drawings are notnecessarily to scale, emphasis instead generally being placed uponillustrating the principles of the invention.

FIG. 1 schematically illustrates a segmented mapping of a lesion.

FIGS. 2 a and 2 b schematically illustrate obtaining cross-sectionalviews of the lesion.

FIG. 3 schematically illustrates using a reference surface as an initialestimate for segmentation.

FIG. 4 schematically illustrates placing estimated surface elements oncross-sections of the lesion in accordance with one embodiment of theinvention.

FIG. 5 a schematically illustrates mapping surface elements of thesegmented image to the estimated surface elements of FIG. 4.

FIG. 5 b schematically illustrates shifting the segmented image inaccordance with the mapped surface elements of the segmented image andthe cross-sectional view of the lesion.

FIG. 6 schematically illustrates placing estimated surface elements on across-section of the lesion in accordance with one embodiment of theinvention.

FIG. 7 is a flow diagram illustrating various embodiments of determininglesion displacement.

FIG. 8 is a schematic illustration of various embodiments of a systemadapted to practice the methods of the present invention.

DETAILED DESCRIPTION

Referring to FIG. 1, an image 100 of a lesion is obtained during atreatment planning stage. The image can be an individual two-dimensionalimage, multiple two-dimensional sectional images or slices thatcollectively represent a volume and may be combined into a compositeimage, or three-dimensional images rendered programmatically or manually(or a combination). The image can be obtained using devices such as CTscanners, ultrasound devices, MRI devices, PET scanners, and x-raydevices, as well as other imaging modalities commonly used throughoutthe art. The structure of interest may be a tumor or lesion independentof other organs, a cancerous gland (such as the prostate) requiringtreatment, a non-cancerous organ of interest, or any identifyinglandmark within the image. The image 100 may be, for instance, used byan oncologist, physician, or radiation technician to determine thelocation and shape of the lesion to be treated and to determine theparameters of the radiation treatment plan such as beam angle, beamshape, the number of beams needed to administer a sufficient dose toeradicate the target lesion, the dose level for each beams, and patientpositioning. However, due to the changes mentioned above, elements inthe image at treatment time are not always in the same location or inthe same shape as they were during the planning stage. Furthermore,because the technicians that administer the treatment generally lack thetraining necessary to build a segmented and/or contoured image of thelesion, these changes are not easily incorporated into an updatedtreatment plan. Therefore, it would be beneficial to be able to use theoriginal segmented image devised by the physician during the treatmentplanning stage during the treatment delivery phase, albeit adjusted forchanges in position and shape. The time required to generate a newsegmented, contoured image is also a constraint as there is limited timeavailable to deliver the individual radiotherapy sessions.

During a treatment planning session, the organ or lesion surface iscontoured into either manually, semi-automatically or automatically anda three-dimensional planning surface image (also referred to herein as a“reference surface”) is generated. The reference surface can bedescribed by points, line segments, a regular mesh, an analyticfunction, a set of triangles or other shapes, or any other surfacerepresentation. A second surface image (referred to as a “treatmentsurface” or “operative surface”) is generated at treatment time. Ineither or both images, some points on the surface may be known with moreconfidence than others due to, for example, poor edge information insome locations. In such a case, each surface element can be assigned aweight that indicates how confident either the user or the segmentationalgorithm is that the surface element corresponds to a true border. Inthe extreme case, a weight of zero indicates that a given surfaceelement is completely arbitrary since there is no reliable image data inthat region.

Still referring to FIG. 1, the image 100 of the lesion to be treatedusing radiation has been segmented into numerous triangular sections,each representing a different surface element. In some embodiments,certain surface elements can be weighted (e.g., elements 105, 110, and115) so as to be given greater consideration by a surface-matchingalgorithm used to identify corresponding elements in a later image(described in greater detail below). Elements 120, 125, and 130 may alsobe considered during a matching process, but to discount anyirregularities or compensate for unknowns in those areas, they can begiven a weighting lower than those assigned to elements 105, 110, and115. In some instances, the imaging modality used to obtain the imagesmay not provide complete representations of the lesion. However,incomplete images can still be used as input into a matching algorithmby weighting surface elements in non-imaged areas with a low, or evenzero weight and assigning a greater weight to elements in areasaccurately characterized in the image. As a result, the algorithm willfind elements of the lesion in the segmented image that correspond tothose elements that are substantially discernible or of greaterimportance. In some cases, each surface element can be assigned an equalweighting, if, for example, weighting is not important for theparticular application.

The weighting of the individual elements can be based on the degree ofcertainty of surface points or elements identified in the image—i.e.,parts of a lesion's surface which are known with a greater degree ofcertainty are assigned higher weights than those about which there isless certainty. The degree of certainty can, for example, represent theaccuracy with which the image or portion thereof accurately correspondsto an actual anatomical feature of the patient; the level of confidencethat the representation is located accurately with respect to otheranatomical features, the patient, fiducial markers, or other positioningdevices; or the degree of detail shown (and accurately represented) inthe image. The weighting of surface elements can also be determinedusing gradient information obtained by analyzing the pixels of the imageto determine closeness to a binary edge. Other reasons for assigninghigher weights to certain surface elements with respect to others may betheir clinical importance, such as their proximity to other criticalorgans or other highly-susceptible non-target tissues. For example, theposterior part of the prostate surface is close to the rectum and thuscan be assigned a higher weight than elements on the rest of theprostate surface to ensure the prostate/rectal interface is correctlyrepositioned.

In addition to determining the weights based on anatomical features ofthe patient, the number and arrangement of the surface elementsthemselves can help determine the assigned weights. For example, theweights can be adjusted for the density of points within a particularregion to reduce bias in the case of an unequal distribution of surfaceelements. Likewise, surface areas with only a few defined elements maybe assigned reduced weights to reduce the effect of statisticaloutliers, to eliminate statistical noise, or to minimize damage to areasthat have not been adequately characterized by imaging.

Technicians generally rely on images such as ultrasound and/or CT todetect changes to the lesion between the planning stage and treatmentstage (and in some cases between individual treatment sessions). Asmentioned above, there may not be sufficient time and/or skill to fullycontour the treatment image prior to delivering the radiation dose. Theinvention provides methods and systems by which these subsequent imagescan be used in conjunction with the original segmented reference imageto produce a segmented image that has effectively been “error-corrected”for lesion shift, rotation and morphisms.

Referring to FIG. 2 a, three-dimensional planning and/or treatmentimages 200 can be separated manually or programmatically into a seriesof two-dimensional cross-sectional image slices for easier manipulation.In one embodiment, images can be the original two-dimensional imagestaken prior to three-dimensional reconstruction. In another embodiment,the cross-sections 205 are radially offset from each other and takenabout a common central axis 210 of the image 200. The degree of offsetand number of images can vary according to individual images andlesions, although ten images offset at an angle of 5 degrees from eachother provides sufficient imaging for a typical prostate gland. Inanother embodiment, and referring to FIG. 2 b, the cross-sections 215are parallel to each other and orthogonal to a common axis 220. Ineither case, the set of two-dimensional slices taken together representsthe three-dimensional image dataset. Segmented structures can also besimilarly divided into two-dimensional curves which are associated toeach two-dimensional cross-sectional slice.

Referring to FIG. 3 a, once the three-dimensional image is acquired andprocessed into a series of cross-sectional two-dimensional frames (300)the lesion or organ 305 is segmented in the treatment image. In oneembodiment, the planning/reference segmentation 310 is superimposed onthe treatment/operative image 300 and moved manually to the approximatelocation of the organ 305. Although only one frame 300 is shown, theuser can also move the planning contour in at least one other orthogonalview plane until it is approximately at the correct position inthree-dimensional space. If, as may be the case, the organ has changedonly in position, then an exact match is possible and the surface can besegmented. If, however, there are morphisms such as changes in volumeand/or shape, the shifted segmentation 310 serves as an “initialestimate” of the organ 305.

Referring to FIG. 3 b, a local segmentation technique is used whichstarts from the initial estimate 310 to deform the contour such that itcontours the organ 305 more accurately. Non-limiting examples of localsegmentation techniques include those that use image gradients toattract contours to edges starting from an estimate of the contour, andtechniques described in “Prostate boundary segmentation from 2Dultrasound images” by Ladak et al. in Med. Phys. 27 (8), p. 1777-1799(2000) and “Prostate boundary segmentation from 3D ultrasound images” byHu et al. in Med. Phys. 30 (7), p. 1648-1659 (2003). The inventionfacilitates the segmentation by using the planning surface, shifted tothe correct location, as the initial estimate. The degree of morphismtolerated depends on the capabilities of the segmentation technique(s)used. In some cases, the initial planning surface can be scaled as wellas shifted so that it represents the lesion in the treatment image moreaccurately. Instead of or in addition to scaling and shifting, theinitial segmentation can be rotated if the lesion has rotated betweenplanning and treatment times.

The local segmentation algorithm used can either be fullythree-dimensional, or can be two-dimensional and applied separately oneach cross-sectional plane 205 or 215. In the latter case, the set oftwo-dimensional curves generated by the segmentation algorithm can bereconstructed into a three-dimensional mesh.

In another embodiment, the step of moving the planning contour manuallyis replaced by an automated process. Once the three-dimensionaltreatment image is acquired, the user defines a number of points on thesurface of the organ to be segmented. The number of points should beminimal to conserve time, but should be sufficient to constrain thesegmentation process in all three dimensions. For instance, referring toFIG. 4, two cross-sectional cuts 400 and 405 through the organ can bepresented to the user. The orientation of the cuts are shown by 410 and415. The user places four points 420 around the cross-section of theorgan 425 in the first cut, and two points 430 in around thecross-section of the organ 435 in the second cut.

Referring to FIGS. 5 a and 5 b, the set of elements 505 from thereference surface can be mapped to corresponding elements 420 and 430identified on the treatment image, and the “shift” 510 necessary to movethe surface image such that the corresponding elements 505 and thepoints 420 and 430 are aligned is measured. The process of detecting andmeasuring the shift can use any point-to-surface matching techniquewhich, for example, minimizes the square distance between the points 420and 430 and the surface, including, but not limited to the weightedsurface-to-surface mapping technique described in currently-pending,co-owned U.S. patent application Ser. No. 10/894,392, the entiredisclosure of which is incorporated by reference herein. The shift 510can be applied to the surface image so that it is “close” to the organof interest in the treatment image. This shifted surface 100′ can thenbe used as an initial estimate to a local segmentation algorithm, aspreviously described, to find the contour of the lesion or organ at timeof treatment.

In some embodiments, such as a clinical setting where only one image maybe presented to the technologist administering the treatment, the userneed only select elements in a single cut through the three-dimensionaltreatment image. Referring to FIG. 6, the cross-sectional slice 600shows an image of the lesion 605 and four user “hint-points” 610 can beindicated by the user on the lesion surface. If the hint points placedby the user in one plane are not sufficient to shift the planningsurface to the correct location in the treatment image as previouslydescribed (because, for example, there are no points to constrain theshift in the direction perpendicular to the imaging plane), additionalestimate points can be automatically identified in other planes using,for example, local segmentation techniques. These techniques can, insome cases, identify a full three-dimensional representation of theorgan. It is unlikely, however, that the resulting contouredrepresentation of the lesion will be suitable because local segmentationtechniques require a good initial estimate, which in general cannot besuitably constructed with hint points in only one plane. The newsurface, or partial surface, or series of points, generated by theinitial local segmentation technique can, however, be used in aconventional point-to-surface or surface-to-surface matching process tobring the planning surface to an approximately correct position in thetreatment image. This shifted planning surface can then, as previouslydescribed, be used as an initial estimate for another pass of the localsegmentation technique to find a better representation of the organ attime of treatment.

In some embodiments, once the lesion or organ is contoured as previouslydescribed, the elements of the surface can once again be used to movethe planning surface to a more accurate location in the treatment image.This process of local segmentation, surface matching, and shifting theplanning surface can be repeated for a number of iterations until theplanning surface is finally at the “best” (i.e., error-minimized)location. Once this best location is determined, then a final run of thelocal segmentation algorithm can be used to identify the edges of theorgan.

One non-limiting example of a segmentation technique used to positionthe reference surface uses landmarks of the lesion that represent strongand/or well-defined edges of the lesion (the base of a prostate gland,for example). The edge points can be identified during an initial run ofthe segmentation process as well as after the user has identified one ormore hint points. These strong edges can be assigned higher weights thanother points, and in one embodiment, any points not in this initial setcan be assigned a weight of zero. Similarly, the rotations orcross-sections used to identify the two-dimensional slices on which theestimate points are identified can be limited to certain rotations(e.g., at 45-degree angles to each other) to avoid capturing falseand/or weak points on the images.

In some cases, segments of the contour obtained at time of planning, thecontour obtained at time of treatment, and/or any intermediary contoursobtained during the iterative process of finding the final treatmentsurface can be assigned a certainty weight c indicating how certain itis that the segmented point corresponds to a true edge. In someembodiments, the certainty weight c is a real number, or in some caseslimited to either 0 or 1 representing uncertain and certain boundaries,respectively. Other methods to generate certainty weights include, butare not limited to, using the magnitude of the gradient at the surfacepoint, a closeness to another edge detection technique, or manualidentification. The result of the segmentation is the generation of asurface of the lesion at the time of treatment planning, S₁. The surfacecan consist of, for example, points, lines, or other surface elementssuch as triangles (e.g., in a mesh). The second image, obtained attreatment time (using, for example a three-dimensional ultrasoundimaging device), S₂, is not segmented, and instead a set (e.g., 4) ofestimated edge points are identified manually for one or morecross-sections of the image. For ease of representation, surfaceelements are hereafter referred to as “points.”

Exemplary Surface-Matching Technique

Various embodiments, such as the one described above, use a surfacematching technique to move the planning surface to an approximatelycorrect location in the treatment image. Thus, techniques which mapsurface elements in a surface S₁ to surface elements in a surface S₂ areused. (Surface elements can be points, triangles, line segments, etc.)Any such technique can be used, but an exemplary technique is asfollows. The set of surface elements and certainty weights on thesegmented surface S₁ are referred to as {r_(j) ⁽¹⁾}={x_(j) ⁽¹⁾, {y_(j)⁽¹⁾, z_(j) ⁽¹⁾, c_(j) ⁽¹⁾} and the estimated surface points of the setof two-dimensional cross-sections S₂ as {r_(i) ⁽²⁾}={x_(i) ⁽²⁾, y_(i)⁽²⁾, z_(i) ⁽²⁾, c_(i) ⁽²⁾}. The index i runs from 1 to the number ofpoints in S₂, and the index j runs from 1 to the number of points N inS₁. The terms x, y, z represent the three-dimensional positionalcoordinates of each point and c refers to the certainty index assignedto that point. Either set of points can be down-sampled to improve thespeed of computation by removing excess points. The set of certaintyindices of S₂ (or, in some cases, S₁ or both S₁ and S₂) may be modifiedto account for the local density of elements in the case of surfacepoints, or the area of the element in the case of a mesh, so that partsof surfaces with large density of elements are not overly weighted inthe algorithm. As a result, the set of elements on S₂ are referred to as{r_(i) ⁽²⁾}={x_(i) ⁽²⁾, y_(i) ⁽²⁾, z_(i) ⁽²⁾, c_(i) ⁽²⁾} where w_(i) ⁽²⁾is the modified set of certainty indices c_(i) ⁽²⁾.

The shift r_(shift) which minimizes least-square error between points onS₁ and S₂ is found, which includes the weights of the points on eachsurface. A method to determine this shift is to minimize the costfunction given by $\begin{matrix}{{C\left( r_{shift} \right)} = {\sum\limits_{i}{W_{i} \cdot {{r_{i}^{(2)} - {r_{close}^{(1)}\left( {{r_{i}^{(2)} - r_{shift}},S_{1}} \right)} - r_{shift}}}^{2}}}} & (1)\end{matrix}$where r_(close) ⁽¹⁾ (r_(i) ⁽²⁾−r_(shift), S₁ refers to the point on S₁which is the closest to the point r_(i) ⁽²⁾−r_(shift), and the weightsW_(i) are defined asW _(i) =w _(i) ⁽²⁾ ·c _(close) ⁽¹⁾(r _(i) ⁽²⁾ −r _(shift) , S ₁)   (2)where c_(close) ⁽¹⁾(r_(i) ⁽²⁾−r_(shift), S₁) refers to the certaintyweight of that closest point on S₁. One suitable method of minimizingthe cost of equation 1 includes the following steps:

-   -   1) Set r_(shift)=(0,0,0)    -   2) Calculate the set of closest points r_(close) ⁽¹⁾ (r_(i)        ⁽²⁾−r_(shift), S₁ in Eq. (1).    -   3) Calculate the cost C(r_(shift)) for this particular shift.    -   4) Is the cost below a threshold? If yes, stop and accept        r_(shift) as the optimal shift. If no, then update r_(shift) to        r_(shift)+ΔR where ΔR is an incremental update step.    -   5) return to step 2) The shift vector r_(shift), is updated by        the update step ΔR until the cost function is a minimum. The        result converges rapidly if ΔR is chosen such that:        $\begin{matrix}        {{\Delta\quad R} = {{\frac{1}{M}{\sum\limits_{i}{W_{i}{r_{close}^{(1)}\left( {{r_{i}^{(2)} - r_{shift}},S_{1}} \right)}}}} - {\frac{1}{N}{\sum\limits_{i}{W_{i}{r_{i}^{(2)}.}}}}}} & (3)        \end{matrix}$        Rotations can also be introduced into the technique.

FIG. 7 illustrates various embodiments of a method of determining anappropriate adjustment to improve the delivery of radiation treatment toa patient. As described above, the process is typically divided into twophases: a treatment planning stage (corresponding to the steps to theleft of the dashed line 700), during which an oncologist or similarlytrained physician prepares a treatment plan for the administration ofradiation to a cancerous lesion; and a treatment delivery stage(corresponding to the steps to the right of the dashed line 700) duringwhich a radiology technician positions the patient within the gantry,makes any adjustments to the positioning based on lesion morphing orshifting, and administers the radiation according to the treatment plan.The treatment planning phase can occur substantially before thetreatment delivery phase, or in some cases immediately preceding thetreatment delivery phase, and may take place in the same room, or insome cases different rooms. As the time span increases between thephases, the target lesion has a greater opportunity to grow, change inmorphology, and change its positioning with respect to surroundingnormal tissue and healthy organs, resulting in a need for positionalcompensation.

As an initial step, a first image of the lesion and surrounding area isobtained (step 705) using any of the imaging modalities described above.In some instances, the image may not be a complete representation of thetarget lesion or surrounding organs, whereas in some instances the imagecan be a very accurate representation of the target area. From thisfirst image, a segmented and/or contoured image consisting of a set ofsurface elements is generated (step 710) representing one or moresurface elements of the lesion and surrounding organs. The set ofsurface elements (collectively the “reference surface”) may include theentire three-dimensional surface, or in some cases where only a portionof the lesion has been accurately imaged, may only describe a portion ofthe lesion. In some embodiments, one or more of the surface elements isweighted based on one or in some cases a combination of the factorsdescribed above. In some embodiments, the reference surface can becontoured manually by a physician on a series of axial CT slices, andthe parallel contours integrated together to form a surface. Thephysician may also use one or more registered and fused datasets to showadditional information, such as MRI, PET or 3D ultrasound. Automaticsegmentation techniques may also aid the physician in segmenting theorgan or lesion of interest.

Subsequent to obtaining the treatment planning image, and inanticipation of a treatment delivery session, a second image of thetarget area is obtained (step 720), using, for example, athree-dimensional ultrasound imaging device. One or more two-dimensionalimage planes are obtained by viewing cross-sections of thethree-dimensional ultrasound image at various angles or positionsrelative to a common axis. A user then indicates estimates of edgepoints within one or more of the two-dimensional images. Any number ofedge points may be identified, but in general, using four points foreach two-dimensional image provides sufficient data to initiate thematching process with the reference surface image. Based on the secondimage, initial estimates of edge points are identified (step 725) and anaxis of rotation is created in the plane of the identified points.Therefore, an approximate initial estimate to the organ surface, or asubset of points on the organ surface, is generated using minimal userinput (e.g., four hint points on only one plane). In some cases, wherean automatic knowledge-based technique is used to makes assumptionsabout anatomical landmarks (e.g., bladder is anterior to the prostatewhich is itself anterior to the rectum), it is possible to dispense withhint points altogether. In some embodiments, multiple two-dimensionalcross-sections can be obtained by rotating about the axis of rotationidentified in the first image, thereby facilitating identification ofadditional edge points. The edge point estimates are used as input to asegmentation process which generates the surface or surface elements ofthe “treatment” surface (step 730). The initial or reference surface canbe shifted, rotated and/or warped (step 735) such that the mappedreference and treatment surface elements are then aligned.

In some cases, and at least for one iteration, the shifted referencesurface can be used as an estimate in subsequent iterations forsegmentation (step 740). An error can then be computed using, forexample, a least-squares fit or other similar convergence calculationapproach that measures the misalignment of the reference surface withthe new treatment surface (step 745) estimates. If the three-dimensionalsurface is properly placed (e.g., a convergence decision threshold ismet in step 750) the process ends, whereas if the error is unacceptable,the process repeats. The number of iterations can be predefined at afixed value, which is known to converge for typical clinical images, orcan be defined, for example, by comparing the amount by which thereference surface is shifted between each iteration and terminating theprocess when the shift falls below a given threshold.

Although the invention is described as relating to a single surface attime of planning and at time of treatment, it may also relate tomultiple lesions and/or organs, some of which are to be treated and someof which are to be avoided.

In step 745, after the three-dimensional treatment surface is found, theplan may be updated to account for the change in location, shape, androtation of the lesion. For example, a completely new plan can bedeveloped, involving changing the beam angles, collimator angles, couchposition, dose weights, etc. using inverse planning algorithms. In someembodiments, the shift can then be translated into a set ofdisplacements for a patient support device, such as a treatment table ofthe LINAC, or the patient, just prior to or during treatment delivery.For example, a shift of (−4, +8) may translate into moving the treatmenttable 4 millimeters to the left and 8 millimeters up with respect to thegantry and radiation beam. The shift can be calculated by finding thecentroid of the current and reference surfaces and shifting thetreatment couch by that amount and/or by implementing a finalsurface-matching algorithm, with or without associated weights asdescribed above, to calculate a couch displacement. Other shifts mayrequire rotating the gantry, moving the patient, or some combinationthereof.

Alternatively, a technician may use simulation techniques to directlymanipulate the patient or patent support device while viewing thereal-time images of the target area on a screen or monitor. In oneembodiment, a technician can adjust the treatment table position withrespect to the LINAC until a desired number of surface elements from thesecond image overlaps corresponding elements in the first image (or viceversa), or a threshold value is reached indicating a sufficient overlap,by manipulating an input device such as a joystick, keypad, or otherinput device. In another embodiment, the technician manually adjusts theposition of the patient on a stationary treatment table until thedesired position is reached. In some cases, the technician may employ acombination of both programmatic adjustments based on pre-determinedalignment displacements and manual patient positioning techniques.

FIG. 8 schematically represents a hardware embodiment of the inventionrealized as a system 800 for determining the changes to a lesion inanticipation of the administration of radiation therapy. The system 800comprises a register 805, a mapping module 810, and a processor 812 andmay optionally include a weighting module 815.

The register 805, which may be any known organized data storage facility(e.g., partitions in RAM, etc.), receives images from an imager 820 suchas an MRI, CT/PET scanner, ultrasound device, or x-ray device. Theregister 805 receives a first image from the imager 820 during orsubsequent to a treatment planning session characterizing the targetregion at the time the treatment plan is determined. The register 805receives a second image from the imager 820 during or just previous to atreatment delivery session characterizing the target region at the timeof treatment. The imaging modalities used during the planning and thetreatment stages can, in some embodiments, be different. In someembodiments, the images can be stored on a data storage device separatefrom the imager (e.g., a database, microfiche, etc.) and sent to thesystem 800. The register 805 may receive the images and beam shapesthrough conventional data ports and may also include circuitry forreceiving analog image data and analog-to-digital conversion circuitryfor digitizing the image data.

The register 805 then determines a set of surface elements from thefirst image either programmatically, or based on some input from theuser. The register 805 optionally provides the image to the weightingmodule 812, which facilitates the assignment of weights to one or moresurface elements generated from the first image. The surface elementsand weights can be determined programmatically, manually, or both. Wheremanual input and manipulation is used, the system 800 receivesinstructions from a user via an input device 830 such as a keyboard, amouse, or other pointing device. Results of the weighting,manipulations, and alignments can also be viewed using a display device840 such as a computer display screen or hand-held device. The set ofsurface elements from the first image, their associated weights, and thesecond image are then sent to the processor 810 which, based on theproximity of the surface elements (and optionally the assigned weights)of the first image and estimated edge points identified on the secondimage, shifts the first image to compensate for the displacement of thelesion as described above. In some embodiments, the processor 815translates displacements into instructions representing physicalmovements of a patient support device 850 and sends the instructions tothe device 850 in order to adjust the position of the patient inaccordance with the alignment calculations.

In some embodiments, the register 805, mapping module 810, weightingmodule 812 and processor 815 may implement the functionality of thepresent invention in hardware or software, or a combination of both on ageneral-purpose computer. In addition, such a program may set asideportions of a computer's random access memory to provide control logicthat affects one or more of the image manipulation, mapping, alignment,and support device control. In such an embodiment, the program may bewritten in any one of a number of high-level languages, such as FORTRAN,PASCAL, C, C++, C#, Java, Tcl, or BASIC. Further, the program can bewritten in a script, macro, or functionality embedded in commerciallyavailable software, such as EXCEL or VISUAL BASIC. Additionally, thesoftware could be implemented in an assembly language directed to amicroprocessor resident on a computer. For example, the software can beimplemented in Intel 80×86 assembly language if it is configured to runon an IBM PC or PC clone. The software may be embedded on an article ofmanufacture including, but not limited to, “computer-readable programmeans” such as a floppy disk, a hard disk, an optical disk, a magnetictape, a PROM, an EPROM, or CD-ROM.

The invention herein is primarily described in relation to external beamradiotherapy, but can be used in any application where a referencesurface is generated at one time and a semi-automatic or fully-automaticsurface is acquired on the same subject at a later time. For example, inbrachytherapy applications a physician often contours a lesion todevelop a preplan, deciding on how many radioactive seeds to purchase,etc. In the operating room, the lesion or organ will require a newsegmentation to develop a new plan, accounting for the new locationand/or shape of the organ. The systems and methods described herein canbe used to segment the lesion in the operating room, using the referencesurface contoured during the preplanning stage.

While the invention has been particularly shown and described withreference to specific embodiments, it should be understood by thoseskilled in the area that various changes in form and detail may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims. The scope of the invention is thusindicated by the appended claims and all changes which come within themeaning and range of equivalency of the claims are therefore intended tobe embraced.

1. A method for obtaining a contoured surface of a lesion within apatient, the method comprising the steps of: (a) providing a contouredreference surface of the lesion based on one or more images acquired ata first time; (b) generating a set of operational images of the lesionat a second time; (c) shifting the reference surface to an estimatedlocation in the operational images; (d) segmenting the operationalimages based at least in part on the shifted reference surface; and (e)determining a contoured surface of the lesion at the second time basedon the segmented images.
 2. The method of claim 1 wherein the shiftingcomprises rotational movement.
 3. The method of claim 1 wherein theshifting comprises translational movement.
 4. The method of claim 1wherein the shifting comprises scaling.
 5. The method of claim 1 whereinthe shifting comprises movement in three dimensions.
 6. The method ofclaim 1 wherein the reference surface is created using one or more CTimages, three-dimensional ultrasound images, two-dimensional ultrasoundimages, PET images and MRI images.
 7. The method of claim 1 wherein theoperational images are obtained using one of a CT scanner, an ultrasounddevice, a PET scanner and an MRI.
 8. The method of claim 1 furthercomprising manually locating the reference surface with respect to theoperational images.
 9. The method of claim 1 further comprisingindicating points within the operational images and mapping the pointsto surface elements on the reference surface to determine the shift. 10.The method of claim 9 wherein the surface elements on the referencesurface comprise at least one of the group consisting of points,triangles, and lines.
 11. The method of claim 9 further comprisingminimizing the distance between the indicated points and mapped surfaceelements.
 12. The method of claim 9 wherein the indicated points are inmultiple planes.
 13. The method of claim 9 wherein the indicated pointsare within one two-dimensional plane.
 14. The method of claim 13 furthercomprising obtaining additional points from the contoured surface andrepeating steps (c) through (d), substituting the shifted referencesurface for the segmented operational images.
 15. The method of claim 1further comprising obtaining two-dimensional cross-sectional curves fromthe contoured reference surface.
 16. The method of claim 15 wherein thetwo-dimensional cross-sectional curves are obtained from the shiftedcontoured reference surface.
 17. The method of claim 15 wherein thetwo-dimensional cross-sectional curves are radially offset from eachother.
 18. The method of claim 15 wherein the two-dimensionalcross-sectional curves are parallel to each other.
 19. The method ofclaim 9 wherein the surface elements on the reference surface representan edge boundary.
 20. The method of claim 9 wherein weights are assignedto the surface elements on the reference surface and the weights arebased at least in part on a clinical importance of an anatomical featurerepresented by the one or more surface elements.
 21. The method of claim20 wherein the clinical importance of the one or more surface elementsis based at least in part on the proximity of an anatomical featurerepresented by the surface element to another anatomical structure ofthe patient.
 22. The method of claim 9 wherein weights are assigned toone or more of the surface elements and the weights are based at leastin part on a density of the one or more surface elements within an areaof the segmented image.
 23. The method of claim 9 wherein the mapping isdetermined by minimizing the mean square distance between surfaceelements in the reference surface and the indicated points.
 24. Themethod of claim 1 wherein the first time is substantially proximate to atreatment planning session.
 25. The method of claim 1 wherein the secondtime is substantially proximate to a treatment session.
 26. A system forobtaining a contoured image of a lesion, the system comprising: (a) aregister for storing (i) a reference surface of the lesion based on oneor more images acquired at a first time, (ii) a set of images of thelesion taken at a second time and (iii) estimated surface elements on atleast one of the set of images; (b) a mapping module for mapping surfaceelements on the reference surface to the estimated surface elements onat least one of the set of images; and (c) a processor for shifting thereference surface based on the matched surface elements.
 27. The systemof claim 26 wherein the reference surface is based on images obtainedusing one of a CT scanner, a three-dimensional ultrasound device, a PETscanner and an MRI.
 28. The system of claim 26 wherein the surfaceelements on the reference surface comprise at least one of the groupconsisting of points, triangles, and lines.
 29. The system of claim 26wherein the estimated surface elements on one of the set of imagescomprise at least one of the group consisting of points, triangles, andlines.
 30. The system of claim 26 wherein the register further obtainstwo-dimensional cross-sectional curves from the shifted referencesurface.
 31. The system of claim 30 wherein the two-dimensionalcross-sectional curves are radially offset from each other.
 32. Thesystem of claim 30 wherein the two-dimensional cross-sectional curvesare parallel to each other.
 33. The system of claim 26 wherein themapping module and processor iteratively map surface elements on thereference surface to surface elements on the shifted reference surface.34. The system of claim 26 further comprising a weighting module forassigning weights to the surface elements on the segmented image. 35.The system of claim 26 wherein the mapping is determined by minimizingthe mean square distance between surface elements on the referencesurface and the one or more estimated surface elements on one of the setof images.
 36. The system of claim 26 wherein the first time issubstantially proximate to a treatment planning session.
 37. The systemof claim 26 wherein the second time is substantially proximate to atreatment session.
 38. An article of manufacture havingcomputer-readable program portions embodied thereon for determining thechanges to a lesion for the purpose of administering radiation treatmentto the lesion, the article comprising computer-readable instructionsfor: (a) providing a contoured surface of the lesion based on one ormore images acquired at a first time as a reference surface; (b)generating a set of images of the lesion at a second time as operationalimages; (c) shifting the reference surface to an estimated location inthe operational images; (d) segmenting the operational images based atleast in part on the shifted reference surface; and (e) determining acontoured surface map of the lesion at the second time based on thesegmented images.